Data-efficient reinforcement learning in continuous state-action Gaussian-POMDPs

We present a data-efficient reinforcement learning method for continuous state-action systems under significant observation noise. Data-efficient solutions under small noise exist, such as PILCO which learns the cartpole swing-up task in 30s. PILCO evaluates policies by planning state-trajectories using a dynamics model. However, PILCO applies policies to the observed state, therefore planning in observation space. We extend PILCO with filtering to instead plan in belief space, consistent with partially observable Markov decisions process (POMDP) planning. This enables data-efficient learning under significant observation noise, outperforming more naive methods such as post-hoc application of a filter to policies optimised by the original (unfiltered) PILCO algorithm. We test our method on the cartpole swing-up task, which involves nonlinear dynamics and requires nonlinear control

Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning

Autonomous vehicle (AV) software is typically composed of a pipeline of individual components, linking sensor inputs to motor outputs. Erroneous component outputs propagate downstream, hence safe AV software must consider the ultimate effect of each component’s errors. Further, improving safety alone is not sufficient. Passengers must also feel safe to trust and use AV systems. To address such concerns, we investigate three under-explored themes for AV research: safety, interpretability, and compliance. Safety can be improved by quantifying the uncertainties of component outputs and propagating them forward through the pipeline. Interpretability is concerned with explaining what the AV observes and why it makes the decisions it does, building reassurance with the passenger. Compliance refers to maintaining some control for the passenger. We discuss open challenges for research within these themes. We highlight the need for concrete evaluation metrics, propose example problems, and highlight possible solutions.Adrian Weller acknowledges support by the Alan Turing Institute under the EPSRC grant EP/N510129/1, and by the Leverhulme Trust via the CFI

Comparison of patients with severe COVID-19 admitted to an intensive care unit in South Africa during the first and second wave of the COVID-19 pandemic

BACKGROUND: The second wave of coronavirus disease 2019 (COVID‑19), dominated by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Beta variant, has been reported to be associated with increased severity in South Africa (SA). OBJECTIVES: To describe and compare clinical characteristics, management and outcomes of COVID‑19 patients admitted to an intensive care unit (ICU) in SA during the first and second waves. METHODS: In a prospective, single-centre, descriptive study, we compared all patients with severe COVID‑19 admitted to ICU during the first and second waves. The primary outcomes assessed were ICU mortality and ICU length of stay (LOS). RESULTS: In 490 patients with comparable ages and comorbidities, no difference in mortality was demonstrated during the second compared with the first wave (65.9% v. 62.5%, p=0.57). ICU LOS was longer in the second wave (10 v. 6 days, p<0.001). More female admissions (67.1% v. 44.6%, p<0.001) and a greater proportion of patients were managed with invasive mechanical ventilation than with non-invasive respiratory support (39.0% v. 14%, p<0.001) in the second wave. CONCLUSIONS: While clinical characteristics were comparable between the two waves, a higher proportion of patients was invasively ventilated and ICU stay was longer in the second. ICU mortality was unchanged

Activation of the dopamine 1 and dopamine 5 receptors increase skeletal muscle mass and force production under non-atrophying and atrophying conditions

<p>Abstract</p> <p>Background</p> <p>Control of skeletal muscle mass and force production is a complex physiological process involving numerous regulatory systems. Agents that increase skeletal muscle cAMP levels have been shown to modulate skeletal muscle mass and force production. The dopamine 1 receptor and its closely related homolog, the dopamine 5 receptor, are G-protein coupled receptors that are expressed in skeletal muscle and increase cAMP levels when activated. Thus we hypothesize that activation of the dopamine 1 and/or 5 receptor will increase skeletal muscle cAMP levels thereby modulating skeletal muscle mass and force production.</p> <p>Methods</p> <p>We treated isolated mouse tibialis anterior (TA) and medial gastrocnemius (MG) muscles in tissue bath with the selective dopamine 1 receptor and dopamine 5 receptor agonist SKF 81297 to determine if activation of skeletal muscle dopamine 1 and dopamine 5 receptors will increase cAMP. We dosed wild-type mice, dopamine 1 receptor knockout mice and dopamine 5 receptor knockout mice undergoing casting-induced disuse atrophy with SKF 81297 to determine if activation of the dopamine 1 and dopamine 5 receptors results in hypertrophy of non-atrophying skeletal muscle and preservation of atrophying skeletal muscle mass and force production.</p> <p>Results</p> <p>In tissue bath, isolated mouse TA and MG muscles responded to SKF 81297 treatment with increased cAMP levels. Treating wild-type mice with SKF 81297 reduced casting-induced TA and MG muscle mass loss in addition to increasing the mass of non-atrophying TA and MG muscles. In dopamine 1 receptor knockout mice, extensor digitorum longus (EDL) and soleus muscle mass and force was not preserved during casting with SKF 81297 treatment, in contrast to significant preservation of casted wild-type mouse EDL and soleus mass and EDL force with SKF 81297 treatment. Dosing dopamine 5 receptor knockout mice with SKF 81297 did not significantly preserve EDL and soleus muscle mass and force although wild-type mouse EDL mass and force was significantly preserved SKF 81297 treatment.</p> <p>Conclusions</p> <p>These data demonstrate for the first time that treatment with a dopamine 1/5 receptor agonist results in (1) significant preservation of EDL, TA, MG and soleus muscle mass and EDL muscle force production during periods of atrophy and (2) hypertrophy of TA and MG muscle. These effects appear to be mainly mediated by both the dopamine 1 and dopamine 5 receptors.</p

Haematological predictors of poor outcome among COVID-19 patients admitted to an intensive care unit of a tertiary hospital in South Africa

BACKGROUND: Studies from Asia, Europe and the USA indicate that widely available haematological parameters could be used to determine the clinical severity of Coronavirus disease 2019 (COVID-19) and predict management outcome. There is limited data from Africa on their usefulness in patients admitted to Intensive Care Units (ICUs). We performed an evaluation of baseline haematological parameters as prognostic biomarkers in ICU COVID-19 patients. METHODS: Demographic, clinical and laboratory data were collected prospectively on patients with confirmed COVID-19, admitted to the adult ICU in a tertiary hospital in Cape Town, South Africa, between March 2020 and February 2021. Robust Poisson regression methods and receiver operating characteristic (ROC) curves were used to explore the association of haematological parameters with COVID-19 severity and mortality. RESULTS: A total of 490 patients (median age 54.1 years) were included, of whom 237 (48%) were female. The median duration of ICU stay was 6 days and 309/490 (63%) patients died. Raised neutrophil count and neutrophil/lymphocyte ratio (NLR) were associated with worse outcome. Independent risk factors associated with mortality were age (ARR 1.01, 95%CI 1.0–1.02; p = 0.002); female sex (ARR 1.23, 95%CI 1.05–1.42; p = 0.008) and D-dimer levels (ARR 1.01, 95%CI 1.002–1.03; p = 0.016). CONCLUSIONS: Our study showed that raised neutrophil count, NLR and D-dimer at the time of ICU admission were associated with higher mortality. Contrary to what has previously been reported, our study revealed females admitted to the ICU had a higher risk of mortality

Effects of etizolam and ethyl loflazepate on the P300 event-related potential in healthy subjects

<p>Abstract</p> <p>Background</p> <p>Benzodiazepines carry the risk of inducing cognitive impairments, which may go unnoticed while profoundly disturbing social activity. Furthermore, these impairments are partly associated with the elimination half-life (EH) of the substance from the body. The object of the present study was to examine the effects of etizolam and ethyl loflazepate, with EHs of 6 h and 122 h, respectively, on information processing in healthy subjects.</p> <p>Methods</p> <p>Healthy people were administered etizolam and ethyl loflazepate acutely and subchronically (14 days). The auditory P300 event-related potential and the neuropsychological batteries described below were employed to assess the effects of drugs on cognition. The P300 event-related potential was recorded before and after drug treatments. The digit symbol test, trail making test, digit span test and verbal paired associates test were administered to examine mental slowing and memory functioning.</p> <p>Results</p> <p>Acute administration of drugs caused prolongation in P300 latency and reduction in P300 amplitude. Etizolam caused a statistically significant prolongation in P300 latency compared to ethyl loflazepate. Furthermore, subchronic administration of etizolam, but not ethyl loflazepate, still caused a weak prolongation in P300 latency. In contrast, neuropsychological tests showed no difference.</p> <p>Conclusions</p> <p>The results indicate that acute administration of ethyl loflazepate induces less effect on P300 latency than etizolam.</p

Dual Infection and Superinfection Inhibition of Epithelial Skin Cells by Two Alphaherpesviruses Co-Occur in the Natural Host

Hosts can be infected with multiple herpesviruses, known as superinfection; however, superinfection of cells is rare due to the phenomenon known as superinfection inhibition. It is believed that dual infection of cells occurs in nature, based on studies examining genetic exchange between homologous alphaherpesviruses in the host, but to date, this has not been directly shown in a natural model. In this report, gallid herpesvirus 2 (GaHV-2), better known as Marek’s disease virus (MDV), was used in its natural host, the chicken, to determine whether two homologous alphaherpesviruses can infect the same cells in vivo. MDV shares close similarities with the human alphaherpesvirus, varicella zoster virus (VZV), with respect to replication in the skin and exit from the host. Recombinant MDVs were generated that express either the enhanced GFP (eGFP) or monomeric RFP (mRFP) fused to the UL47 (VP13/14) herpesvirus tegument protein. These viruses exhibited no alteration in pathogenic potential and expressed abundant UL47-eGFP or -mRFP in feather follicle epithelial cells in vivo. Using laser scanning confocal microscopy, it was evident that these two similar, but distinguishable, viruses were able to replicate within the same cells of their natural host. Evidence of superinfection inhibition was also observed. These results have important implications for two reasons. First, these results show that during natural infection, both dual infection of cells and superinfection inhibition can co-occur at the cellular level. Secondly, vaccination against MDV with homologous alphaherpesvirus like attenuated GaHV-2, or non-oncogenic GaHV-3 or meleagrid herpesvirus (MeHV-1) has driven the virus to greater virulence and these results implicate the potential for genetic exchange between homologous avian alphaherpesviruses that could drive increased virulence. Because the live attenuated varicella vaccine is currently being administered to children, who in turn could be superinfected by wild-type VZV, this could potentiate recombination events of VZV as well

Within-Host Dynamics of the Hepatitis C Virus Quasispecies Population in HIV-1/HCV Coinfected Patients

HIV/HCV coinfected individuals under highly active antiretroviral therapy (HAART) represent an interesting model for the investigation of the role played by the immune system in driving the evolution of the HCV quasispecies. We prospectively studied the intra-host evolution of the HCV heterogeneity in 8 coinfected subjects, selected from a cohort of 32 patients initiating HAART: 5 immunological responders (group A) and 3 immunological non-responders (group B), and in two HCV singly infected controls not assuming drugs (group C). For all these subjects at least two serial samples obtained at the first observation (before HAART) and more than 1 year later, underwent clonal sequence analysis of partial E1/E2 sequences, encompassing the whole HVR1. Evolutionary rates, dated phylogenies and population dynamics were co-estimated by using a Bayesian Markov Chain Monte Carlo approach, and site specific selection pressures were estimated by maximum likelihood-based methods. The intra-host evolutionary rates of HCV quasispecies was 10 times higher in subjects treated with HAART than in controls without immunodeficiency (1.9 and 2.3×10−3 sub/site/month in group A and B and 0.29×10−3 sub/site/month in group C individuals). The within-host Bayesian Skyline plot analysis showed an exponential growth of the quasispecies populations in immunological responders, coinciding with a peak in CD4 cell counts. On the contrary, quasispecies population remained constant in group B and in group C controls. A significant positive selection pressure was detected in a half of the patients under HAART and in none of the group C controls. Several sites under significant positive selection were described, mainly included in the HVR1. Our data indicate that different forces, in addition to the selection pressure, drive an exceptionally fast evolution of HCV during HAART immune restoration. We hypothesize that an important role is played by the enlargement of the viral replicative space

The Actin Binding Domain of βI-Spectrin Regulates the Morphological and Functional Dynamics of Dendritic Spines

Actin microfilaments regulate the size, shape and mobility of dendritic spines and are in turn regulated by actin binding proteins and small GTPases. The βI isoform of spectrin, a protein that links the actin cytoskeleton to membrane proteins, is present in spines. To understand its function, we expressed its actin-binding domain (ABD) in CA1 pyramidal neurons in hippocampal slice cultures. The ABD of βI-spectrin bundled actin in principal dendrites and was concentrated in dendritic spines, where it significantly increased the size of the spine head. These effects were not observed after expression of homologous ABDs of utrophin, dystrophin, and α-actinin. Treatment of slice cultures with latrunculin-B significantly decreased spine head size and decreased actin-GFP fluorescence in cells expressing the ABD of α-actinin, but not the ABD of βI-spectrin, suggesting that its presence inhibits actin depolymerization. We also observed an increase in the area of GFP-tagged PSD-95 in the spine head and an increase in the amplitude of mEPSCs at spines expressing the ABD of βI-spectrin. The effects of the βI-spectrin ABD on spine size and mEPSC amplitude were mimicked by expressing wild-type Rac3, a small GTPase that co-immunoprecipitates specifically with βI-spectrin in extracts of cultured cortical neurons. Spine size was normal in cells co-expressing a dominant negative Rac3 construct with the βI-spectrin ABD. We suggest that βI-spectrin is a synaptic protein that can modulate both the morphological and functional dynamics of dendritic spines, perhaps via interaction with actin and Rac3